A Vietoris-Smale mapping theorem for the homotopy of hyperdefinable sets

Results of Smale (1957) and Dugundji (1969) allow to compare the homotopy groups of two topological spaces $X$ and $Y$ whenever a map $f:X\to Y$ with strong connectivity conditions on the fibers is given. We apply similar techniques in o-minimal expansions of fields to compare the o-minimal homotopy of a definable set $X$ with the homotopy of some of its bounded hyperdefinable quotients $X/E$. Under suitable assumption, we show that $\pi_{n}(X)^{\rm def}\cong\pi_{n}(X/E)$ and $\dim(X)=\dim_{\mathbb R}(X/E)$. As a special case, given a definably compact group, we obtain a new proof of Pillay's group conjecture "$\dim(G)=\dim_{\mathbb R}(G/G^{00}$)" largely independent of the group structure of $G$. We also obtain different proofs of various comparison results between classical and o-minimal homotopy.Comment: 24 page

Emergence of Invariance and Disentanglement in Deep Representations

Using established principles from Statistics and Information Theory, we show that invariance to nuisance factors in a deep neural network is equivalent to information minimality of the learned representation, and that stacking layers and injecting noise during training naturally bias the network towards learning invariant representations. We then decompose the cross-entropy loss used during training and highlight the presence of an inherent overfitting term. We propose regularizing the loss by bounding such a term in two equivalent ways: One with a Kullbach-Leibler term, which relates to a PAC-Bayes perspective; the other using the information in the weights as a measure of complexity of a learned model, yielding a novel Information Bottleneck for the weights. Finally, we show that invariance and independence of the components of the representation learned by the network are bounded above and below by the information in the weights, and therefore are implicitly optimized during training. The theory enables us to quantify and predict sharp phase transitions between underfitting and overfitting of random labels when using our regularized loss, which we verify in experiments, and sheds light on the relation between the geometry of the loss function, invariance properties of the learned representation, and generalization error.Comment: Deep learning, neural network, representation, flat minima, information bottleneck, overfitting, generalization, sufficiency, minimality, sensitivity, information complexity, stochastic gradient descent, regularization, total correlation, PAC-Baye

The Information Complexity of Learning Tasks, their Structure and their Distance

We introduce an asymmetric distance in the space of learning tasks, and a framework to compute their complexity. These concepts are foundational for the practice of transfer learning, whereby a parametric model is pre-trained for a task, and then fine-tuned for another. The framework we develop is non-asymptotic, captures the finite nature of the training dataset, and allows distinguishing learning from memorization. It encompasses, as special cases, classical notions from Kolmogorov complexity, Shannon, and Fisher Information. However, unlike some of those frameworks, it can be applied to large-scale models and real-world datasets. Our framework is the first to measure complexity in a way that accounts for the effect of the optimization scheme, which is critical in Deep Learning

Thromboprophylaxis in day surgery

AbstractMany patients undergoing day surgery are at low-risk of venous thromboembolic events. However, given that pulmonary embolism is the most common preventable cause of hospital death, the risk-benefit profile of thromboprophylaxis should be accurately balanced. In this narrative review, we will briefly discuss some topics of thromboprohylaxis in ambulatory surgical procedures: venous thromboembolic risk stratification, venous thromboembolic risk during laparoscopic surgery, use of antithrombotic drugs in case of neuraxial anesthesia/analgesia, American College of Chest Physicians recommendations for thromboprophylaxis

Critical Learning Periods for Multisensory Integration in Deep Networks

We show that the ability of a neural network to integrate information from diverse sources hinges critically on being exposed to properly correlated signals during the early phases of training. Interfering with the learning process during this initial stage can permanently impair the development of a skill, both in artificial and biological systems where the phenomenon is known as critical learning period. We show that critical periods arise from the complex and unstable early transient dynamics, which are decisive of final performance of the trained system and their learned representations. This evidence challenges the view, engendered by analysis of wide and shallow networks, that early learning dynamics of neural networks are simple, akin to those of a linear model. Indeed, we show that even deep linear networks exhibit critical learning periods for multi-source integration, while shallow networks do not. To better understand how the internal representations change according to disturbances or sensory deficits, we introduce a new measure of source sensitivity, which allows us to track the inhibition and integration of sources during training. Our analysis of inhibition suggests cross-source reconstruction as a natural auxiliary training objective, and indeed we show that architectures trained with cross-sensor reconstruction objectives are remarkably more resilient to critical periods. Our findings suggest that the recent success in self-supervised multi-modal training compared to previous supervised efforts may be in part due to more robust learning dynamics and not solely due to better architectures and/or more data